Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation.

Supervised deep learning-based methods yield accurate results for medical image segmentation. However, they require large labeled datasets for this, and obtaining them is a laborious task that requires clinical expertise. Semi/self-supervised learning-based approaches address this limitation by exploiting unlabeled data along with limited annotated data. Recent self-supervised learning methods use contrastive loss to learn good global level representations from unlabeled images and achieve high performance in classification tasks on popular natural image datasets like ImageNet. In pixel-level prediction tasks such as segmentation, it is crucial to also learn good local level representations along with global representations to achieve better accuracy. However, the impact of the existing local contrastive loss-based methods remains limited for learning good local representations because similar and dissimilar local regions are defined based on random augmentations and spatial proximity; not based on the semantic label of local regions due to lack of large-scale expert annotations in the semi/self-supervised setting. In this paper, we propose a local contrastive loss to learn good pixel level features useful for segmentation by exploiting semantic label information obtained from pseudo-labels of unlabeled images alongside limited annotated images with ground truth (GT) labels. In particular, we define the proposed contrastive loss to encourage similar representations for the pixels that have the same pseudo-label/GT label while being dissimilar to the representation of pixels with different pseudo-label/GT label in the dataset. We perform pseudo-label based self-training and train the network by jointly optimizing the proposed contrastive loss on both labeled and unlabeled sets and segmentation loss on only the limited labeled set. We evaluated the proposed approach on three public medical datasets of cardiac and prostate anatomies, and obtain high segmentation performance with a limited labeled set of one or two 3D volumes. Extensive comparisons with the state-of-the-art semi-supervised and data augmentation methods and concurrent contrastive learning methods demonstrate the substantial improvement achieved by the proposed method. The code is made publicly available at https://github.com/krishnabits001/pseudo_label_contrastive_training.

Sampling Possible Reconstructions of Undersampled Acquisitions in MR Imaging With a Deep Learned Prior.

Undersampling the k-space during MR acquisitions saves time, however results in an ill-posed inversion problem, leading to an infinite set of images as possible solutions. Traditionally, this is tackled as a reconstruction problem by searching for a single "best" image out of this solution set according to some chosen regularization or prior. This approach, however, misses the possibility of other solutions and hence ignores the uncertainty in the inversion process. In this paper, we propose a method that instead returns multiple images which are possible under the acquisition model and the chosen prior to capture the uncertainty in the inversion process. To this end, we introduce a low dimensional latent space and model the posterior distribution of the latent vectors given the acquisition data in k-space, from which we can sample in the latent space and obtain the corresponding images. We use a variational autoencoder for the latent model and the Metropolis adjusted Langevin algorithm for the sampling. We evaluate our method on two datasets; with images from the Human Connectome Project and in-house measured multi-coil images. We compare to five alternative methods. Results indicate that the proposed method produces images that match the measured k-space data better than the alternatives, while showing realistic structural variability. Furthermore, in contrast to the compared methods, the proposed method yields higher uncertainty in the undersampled phase encoding direction, as expected.

Semi-supervised task-driven data augmentation for medical image segmentation.

Supervised learning-based segmentation methods typically require a large number of annotated training data to generalize well at test time. In medical applications, curating such datasets is not a favourable option because acquiring a large number of annotated samples from experts is time-consuming and expensive. Consequently, numerous methods have been proposed in the literature for learning with limited annotated examples. Unfortunately, the proposed approaches in the literature have not yet yielded significant gains over random data augmentation for image segmentation, where random augmentations themselves do not yield high accuracy. In this work, we propose a novel task-driven data augmentation method for learning with limited labeled data where the synthetic data generator, is optimized for the segmentation task. The generator of the proposed method models intensity and shape variations using two sets of transformations, as additive intensity transformations and deformation fields. Both transformations are optimized using labeled as well as unlabeled examples in a semi-supervised framework. Our experiments on three medical datasets, namely cardiac, prostate and pancreas, show that the proposed approach significantly outperforms standard augmentation and semi-supervised approaches for image segmentation in the limited annotation setting. The code is made publicly available at https://github.com/krishnabits001/task_driven_data_augmentation.

Test-time adaptable neural networks for robust medical image segmentation.

Convolutional Neural Networks (CNNs) work very well for supervised learning problems when the training dataset is representative of the variations expected to be encountered at test time. In medical image segmentation, this premise is violated when there is a mismatch between training and test images in terms of their acquisition details, such as the scanner model or the protocol. Remarkable performance degradation of CNNs in this scenario is well documented in the literature. To address this problem, we design the segmentation CNN as a concatenation of two sub-networks: a relatively shallow image normalization CNN, followed by a deep CNN that segments the normalized image. We train both these sub-networks using a training dataset, consisting of annotated images from a particular scanner and protocol setting. Now, at test time, we adapt the image normalization sub-network for each test image, guided by an implicit prior on the predicted segmentation labels. We employ an independently trained denoising autoencoder (DAE) in order to model such an implicit prior on plausible anatomical segmentation labels. We validate the proposed idea on multi-center Magnetic Resonance imaging datasets of three anatomies: brain, heart and prostate. The proposed test-time adaptation consistently provides performance improvement, demonstrating the promise and generality of the approach. Being agnostic to the architecture of the deep CNN, the second sub-network, the proposed design can be utilized with any segmentation network to increase robustness to variations in imaging scanners and protocols. Our code is available at: https://github.com/neerakara/test-time-adaptable-neural-networks-for-domain-generalization.

An image interpolation approach for acquisition time reduction in navigator-based 4D MRI.

Navigated 2D multi-slice dynamic Magnetic Resonance (MR) imaging enables high contrast 4D MR imaging during free breathing and provides in-vivo observations for treatment planning and guidance. Navigator slices are vital for retrospective stacking of 2D data slices in this method. However, they also prolong the acquisition sessions. Temporal interpolation of navigator slices can be used to reduce the number of navigator acquisitions without degrading specificity in stacking. In this work, we propose a convolutional neural network (CNN) based method for temporal interpolation, with motion field prediction as an intermediate step. The proposed formulation incorporates the prior knowledge that a motion field underlies changes in the image intensities over time. Previous approaches that interpolate directly in the intensity space are prone to produce blurry images or even remove structures in the images. Our method avoids such problems and faithfully preserves the information in the image. Further, an important advantage of our formulation is that it provides an unsupervised estimation of bi-directional motion fields. These motion fields can potentially be used to halve the number of registrations required during 4D reconstruction, thus substantially reducing the reconstruction time. These advantages are achieved while preserving 4D reconstruction quality as compared to that with the true navigators.

Reducing Navigators in Free-Breathing Abdominal MRI via Temporal Interpolation Using Convolutional Neural Networks.

Navigated 2-D multi-slice dynamic magnetic resonance imaging (MRI) acquisitions are essential for MR guided therapies. This technique yields time-resolved volumetric images during free-breathing, which are ideal for visualizing and quantifying breathing induced motion. To achieve this, navigated dynamic imaging requires acquiring multiple navigator slices. Reducing the number of navigator slices would allow for acquiring more data slices in the same time, and hence, increasing through-plane resolution or alternatively the overall acquisition time can be reduced while keeping resolution unchanged. To this end, we propose temporal interpolation of navigator slices using convolutional neural networks (CNNs). Our goal is to acquire fewer navigators and replace the missing ones with interpolation. We evaluate the proposed method on abdominal navigated dynamic MRI sequences acquired from 14 subjects. Investigations with several CNN architectures and training loss functions show favorable results for cost and a simple feed-forward network with no skip connections. When compared with interpolation by non-linear registration, the proposed method achieves higher interpolation accuracy on average as quantified in terms of root mean square error and residual motion. Analysis of the differences shows that the better performance is due to more accurate interpolation at peak exhalation and inhalation positions. Furthermore, the CNN-based approach requires substantially lower execution times than that of the registration-based method. At last, experiments on dynamic volume reconstruction reveal minimal differences between reconstructions with acquired and interpolated navigator slices.

Interactive segmentation in MRI for orthopedic surgery planning: bone tissue.

PURPOSE: Planning orthopedic surgeries is commonly performed in computed tomography (CT) images due to the higher contrast of bony structure. However, soft tissues such as muscles and ligaments that may determine the functional outcome of a procedure are not easy to identify in CT, for which fast and accurate segmentation in MRI would be desirable. To be usable in daily practice, such method should provide convenient means of interaction for modifications and corrections, e.g., during perusal by the surgeon or the planning physician for quality control. METHODS: We propose an interactive segmentation framework for MR images and evaluate the outcome for segmentation of bones. We use a random forest classification and a random walker-based spatial regularization. The latter enables the incorporation of user input as well as enforcing a single connected anatomical structures, thanks to which a selective sampling strategy is proposed to substantially improve the supervised learning performance. RESULTS: We evaluated our segmentation framework on 10 patient humerus MRI as well as 4 high-resolution MRI from volunteers. Interactive humerus segmentations for patients took on average 150 s with over 3.5 times time-gain compared to manual segmentations, with accuracies comparable (converging) to that of much longer interactions. For high-resolution data, a novel multi-resolution random walker strategy further reduced the run time over 20 times of the manual segmentation, allowing for a feasible interactive segmentation framework. CONCLUSIONS: We present a segmentation framework that allows iterative corrections leading to substantial speed gains in bone annotation in MRI. This will allow us to pursue semi-automatic segmentations of other musculoskeletal anatomy first in a user-in-the-loop manner, where later less user interactions or perhaps only few for quality control will be necessary as our annotation suggestions improve.